{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "78d8ee28",
   "metadata": {},
   "source": [
    "# 第六节：保存与加载\n",
    "顺着官方文档，我们来学习如何保存模型参数、加载模型参数和查看正确率，首先我们训练一个模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "0aabf609",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] ME(23976:23868,MainProcess):2022-10-01-21:29:01.140.714 [mindspore\\dataset\\engine\\datasets_user_defined.py:656] Python multiprocessing is not supported on Windows platform.\n",
      "[WARNING] ME(23976:23868,MainProcess):2022-10-01-21:29:01.144.722 [mindspore\\dataset\\core\\validator_helpers.py:804] 'Resize' from mindspore.dataset.vision.c_transforms is deprecated from version 1.8 and will be removed in a future version. Use 'Resize' from mindspore.dataset.vision instead.\n",
      "[WARNING] ME(23976:23868,MainProcess):2022-10-01-21:29:01.145.757 [mindspore\\dataset\\core\\validator_helpers.py:804] 'Rescale' from mindspore.dataset.vision.c_transforms is deprecated from version 1.8 and will be removed in a future version. Use 'Rescale' from mindspore.dataset.vision instead.\n",
      "[WARNING] ME(23976:23868,MainProcess):2022-10-01-21:29:01.146.755 [mindspore\\dataset\\core\\validator_helpers.py:804] 'Rescale' from mindspore.dataset.vision.c_transforms is deprecated from version 1.8 and will be removed in a future version. Use 'Rescale' from mindspore.dataset.vision instead.\n",
      "[WARNING] ME(23976:23868,MainProcess):2022-10-01-21:29:01.147.716 [mindspore\\dataset\\core\\validator_helpers.py:804] 'HWC2CHW' from mindspore.dataset.vision.c_transforms is deprecated from version 1.8 and will be removed in a future version. Use 'HWC2CHW' from mindspore.dataset.vision instead.\n"
     ]
    }
   ],
   "source": [
    "import mindspore as ms\n",
    "import mindspore.nn as nn\n",
    "from mindvision.classification.dataset import Mnist\n",
    "from mindvision.classification.models import LeNet5\n",
    "from mindspore.nn import Momentum\n",
    "from mindvision.engine.callback import LossMonitor\n",
    "# 定义超参\n",
    "epochs = 10\n",
    "batch_size = 32\n",
    "lr = 0.01\n",
    "momentum = 0.9\n",
    "# 加载数据集\n",
    "dir = \"./Mnist\"\n",
    "dataset = Mnist(path=dir, batch_size=batch_size, shuffle=True, split='train', repeat_num=1)\n",
    "dataset = dataset.run()\n",
    "# 定义网络\n",
    "net = LeNet5(num_classes=10)\n",
    "# 定义损失函数\n",
    "loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n",
    "# 定义优化器\n",
    "optim = Momentum(net.trainable_params(), learning_rate=lr, momentum=momentum)\n",
    "# 初始化模型\n",
    "model = ms.Model(net, loss_fn=loss, optimizer=optim, metrics={'acc'})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10c57861",
   "metadata": {},
   "source": [
    "## 一、保存模型\n",
    "使用mindspopre提供的`save_checkpoint`接口可直接保存模型，参数为模型和保存路径。这种保存方式只会保存当前模型的参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c653bdf9",
   "metadata": {},
   "outputs": [],
   "source": [
    "ms.save_checkpoint(net, \"./LeNet5/save\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51229962",
   "metadata": {},
   "source": [
    "更多的情况下，我们是在模型训练的过程中保存模型的参数，使用mindspore中的`CheckpointConfig`和`ModelCheckpoint`来自动保存模型参数，生成ckpt文件。\n",
    "使用`CheckpointConfig`设置模型保存的参数，`save_checkpoint_steps`表示每隔多少个step保存一次，`keep_checkpoint_max`表示最多保留参数文件的数量。\n",
    "使用`ModelCheckpoint`保存模型参数，`prefix`表示模型参数文件的名字，`directory`表示模型参数文件保存的路径。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ae588e83",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch:[  0/ 10], step:[ 1875/ 1875], loss:[0.195/1.072], time:21.024 ms, lr:0.01000\n",
      "Epoch time: 10361.203 ms, per step time: 5.526 ms, avg loss: 1.072\n",
      "Epoch:[  1/ 10], step:[ 1875/ 1875], loss:[0.022/0.077], time:17.920 ms, lr:0.01000\n",
      "Epoch time: 10349.641 ms, per step time: 5.520 ms, avg loss: 0.077\n",
      "Epoch:[  2/ 10], step:[ 1875/ 1875], loss:[0.013/0.050], time:5.097 ms, lr:0.01000\n",
      "Epoch time: 10822.644 ms, per step time: 5.772 ms, avg loss: 0.050\n",
      "Epoch:[  3/ 10], step:[ 1875/ 1875], loss:[0.017/0.039], time:23.258 ms, lr:0.01000\n",
      "Epoch time: 10661.460 ms, per step time: 5.686 ms, avg loss: 0.039\n",
      "Epoch:[  4/ 10], step:[ 1875/ 1875], loss:[0.056/0.031], time:0.000 ms, lr:0.01000\n",
      "Epoch time: 10650.284 ms, per step time: 5.680 ms, avg loss: 0.031\n",
      "Epoch:[  5/ 10], step:[ 1875/ 1875], loss:[0.013/0.026], time:19.011 ms, lr:0.01000\n",
      "Epoch time: 11133.037 ms, per step time: 5.938 ms, avg loss: 0.026\n",
      "Epoch:[  6/ 10], step:[ 1875/ 1875], loss:[0.003/0.022], time:18.480 ms, lr:0.01000\n",
      "Epoch time: 11023.245 ms, per step time: 5.879 ms, avg loss: 0.022\n",
      "Epoch:[  7/ 10], step:[ 1875/ 1875], loss:[0.013/0.018], time:15.655 ms, lr:0.01000\n",
      "Epoch time: 10964.938 ms, per step time: 5.848 ms, avg loss: 0.018\n",
      "Epoch:[  8/ 10], step:[ 1875/ 1875], loss:[0.025/0.017], time:16.035 ms, lr:0.01000\n",
      "Epoch time: 10897.529 ms, per step time: 5.812 ms, avg loss: 0.017\n",
      "Epoch:[  9/ 10], step:[ 1875/ 1875], loss:[0.002/0.014], time:15.628 ms, lr:0.01000\n",
      "Epoch time: 10886.325 ms, per step time: 5.806 ms, avg loss: 0.014\n"
     ]
    }
   ],
   "source": [
    "# 设置模型保存参数\n",
    "config = ms.CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10)\n",
    "# 应用模型保存参数\n",
    "ckpt = ms.ModelCheckpoint(prefix=\"LeNet5\", directory=\"./LeNet5\", config=config)\n",
    "model.train(epochs, dataset, callbacks=[ckpt, LossMonitor(lr, 1875)]) # callback有多个参数时要加s"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cce35bd8",
   "metadata": {},
   "source": [
    "## 二、加载模型\n",
    "我们可以使用mindspore中的`load_checkpoint`和`load_param_into_net`方法来加载模型参数。`load_checkpoint`方法返回参数字典，方法参数为ckpt文件路径；`load_param_into_net`方法将参数加载到神经网络中，方法参数为神经网络和参数字典。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "5a9827ec",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 生成参数字典\n",
    "param_dict = ms.load_checkpoint(\"./leNet5/LeNet5-10_1875.ckpt\")\n",
    "# 重新定义一个网络\n",
    "net = LeNet5()\n",
    "# 将参数加载到网络\n",
    "ms.load_param_into_net(net, param_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "314045a5",
   "metadata": {},
   "source": [
    "## 三、模型验证\n",
    "使用`eval`接口进行推理验证，查看正确率，`eval`方法会返回正确率，参数为数据集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "53f34934",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] ME(23976:23868,MainProcess):2022-10-01-21:32:57.298.656 [mindspore\\dataset\\engine\\datasets_user_defined.py:656] Python multiprocessing is not supported on Windows platform.\n",
      "[WARNING] ME(23976:23868,MainProcess):2022-10-01-21:32:57.298.656 [mindspore\\dataset\\core\\validator_helpers.py:804] 'Resize' from mindspore.dataset.vision.c_transforms is deprecated from version 1.8 and will be removed in a future version. Use 'Resize' from mindspore.dataset.vision instead.\n",
      "[WARNING] ME(23976:23868,MainProcess):2022-10-01-21:32:57.298.656 [mindspore\\dataset\\core\\validator_helpers.py:804] 'Rescale' from mindspore.dataset.vision.c_transforms is deprecated from version 1.8 and will be removed in a future version. Use 'Rescale' from mindspore.dataset.vision instead.\n",
      "[WARNING] ME(23976:23868,MainProcess):2022-10-01-21:32:57.298.656 [mindspore\\dataset\\core\\validator_helpers.py:804] 'Rescale' from mindspore.dataset.vision.c_transforms is deprecated from version 1.8 and will be removed in a future version. Use 'Rescale' from mindspore.dataset.vision instead.\n",
      "[WARNING] ME(23976:23868,MainProcess):2022-10-01-21:32:57.298.656 [mindspore\\dataset\\core\\validator_helpers.py:804] 'HWC2CHW' from mindspore.dataset.vision.c_transforms is deprecated from version 1.8 and will be removed in a future version. Use 'HWC2CHW' from mindspore.dataset.vision instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'acc': 0.9898838141025641}\n"
     ]
    }
   ],
   "source": [
    "# 加载测试数据集\n",
    "dataset = Mnist(path=dir, shuffle=True, batch_size=32, split='test', download=True)\n",
    "dataset = dataset.run()\n",
    "# 使用eval函数进行推理验证,查看正确率\n",
    "acc = model.eval(dataset)\n",
    "print(f\"{acc}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mindspore",
   "language": "python",
   "name": "mindvision"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
